Non-Parametric Statistical Techniques for Computational Forensic Engineering

Computational forensic engineering is the process of identification of the tool or algorithm that was used to produce a particular output or solution by examining the structural properties of the output. We introduce a new Relative Generic Forensic Engineering (RGFE) technique that has several advantages over the previously proposed approaches. From the quantitative point of view, the new RGFE technique performs not only more accurate identification of the tool used but also provides the identification with a level of confidence. A higher degree of classification is achieved by our technique with the ability to identify the output as produced by an unknown tool. We introduce a generic formulation which enables rapid application of the RGFE approach to a variety of problems that can be formulated as 0-1 integer linear programs. Additionally, we present forensic engineering scenarios which enable a natural classification of the forensic engineering task with respect to the types and amount of information available to conduct the classification. From the technical point of view, the key innovations of the RGFE technique include the development of a simulated annealing-based CART classification and clustering technique and a generic property formulation technique which provides a systematic way to develop properties for a given problem or facilitates their reuse. In addition to solution properties, we introduce instance properties which enable an enhanced classification of problem instances leading to a higher accuracy of algorithm identification. Finally, the single most important innovation, property calibration, interprets the value for a given algorithm for a given property relative to the values for other algorithms. We demonstrated the RGFE technique on two canonical optimization problems: boolean satisfiability (SAT) and graph coloring (GC) and used statistical techniques to establish the effectiveness of the approach.